Perspective Invariant Movie Analysis for Depth Recovery

Processing entire movies and taking advantage of the inter-frame redundancy is the key of shape-from-motion analysis. Thus, recovering the depth of a xed scene from an image sequence can be viewed as a movie processing problem : how to focus the redundant depth information of a noisy image sequence into a perfect depth-coherent movie ? We present a natural set of axioms in agreement with the depth recovery, in the simple case of a straight movement of the camera parallel to the focal plane. According to these axioms, we show that there is a unique depth-coherent way of processing movies, described by a nonlinear partial diierential equation. The corresponding multiscale analysis has the property of smoothing the motion eld of a movie, leading naturally to a perfect motion eld compatible with a depth interpretation. Moreover, in the case of an ideal movie, i.e. coherent with the observation of a xed 3D scene, this analysis can be viewed as a simple ltering of the camera movement preserving the depth interpretation given by the movie, and is thereby perspective invariant. Last, we study a numerical scheme, compatible with the theoritical axioms, and produce some experiments on synthetic noisy movies.